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2a)D This action is FINAL. 2b)H This action is non-final. 
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4) M Claim(s) 1-18 is/are pending in the application. 
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6) H Claim(s) 1-18 is/are rejected. 
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Application Papers 
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10) ^ The drawing(s) filed on 13 February 2002 is/are: a)[3 accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 
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2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
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DETAILED ACTION 

Claim Rejections - 35 USC § 101 

35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. Claim 10 recites a computer program product 
to perform the method of claim 1 . It is the Office's position that this is non-statutory. A 
computer program product must be embodied in a computer readable medium. 
Appropriate correction is required. 

Claim Rejections - 35 USC §112 

The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

Claims 5 and 15 are rejected under 35 U.S.C. 1 12, first paragraph, as based on 
a disclosure which is not enabling. The claimed the element of a color moment, which is 
critical or essential to the practice of the invention, but not included in the claim(s) is not 
enabled by the disclosure. See In re Mayhew, 527 F.2d 1229, 188 USPQ 356 (CCPA 
1976). The phrase "color moment" does not appear in anywhere in the specification. It 
is also unclear what a color moment is with no definition given in the specification and a 
1 12 2 nd paragraph rejection is also cited below. Appropriate correction is required. 
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The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

Claims 5 and 15 are rejected under 35 U.S.C. 112, second paragraph, as being 

indefinite for failing to particularly point out and distinctly claim the subject matter which 

applicant regards as the invention. The claimed element of a "color moment" does not 

appear anywhere in the specification and further it is unclear what a color moment might 

be as moment is typically used to describe a point in time and it is unclear how color 

describes a moment or vice versa. 

Claims 1-18 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. In particular independent claims 1 and 1 1 use the 
word "semantic" to describe an object, which can be detected and from which, some 
kind of orientation can be gathered. The only definitions of semantic seemed to be 
directed to language or words. 

From dictionary.com - definition of Semantic: 

1. Of or relating to meaning, especially meaning in language. 

2. Of, relating to, or according to the science of semantics. 

In the environment of the present invention, semantic is being used to describe 
an object as a distinctive object from which orientation can be determined. So when 
applicant uses text as a semantic object the definition makes some sense, but when 
applicant uses for example in claim 3, human face, human figure, clear blue sky, lawn 
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grass, a snow field, body of water, tree, a sign, and written text it is unclear how these 
objects are "semantic." Appropriate correction is required. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

Claims 1-5, 8-15 and 18 are rejected under 35 U.S.C. 102(e) as being 
anticipated by U.S. Patent 6,798,905 to Sugiura et al. 

With regard to claim 1 , Sugiura discloses a method for determining the 
orientation of a digital image, comprising the steps of: 

a) employing a semantic object detection method to detect the presence and 
orientation of a semantic object (column 9, lines 20-25). Sugiura discloses using text as 
a semantic object. The orientation of the text is calculated and then a "line direction" or 
orientation is determined. 

b) employing a scene layout detection method to detect the orientation of a 
scene layout (column 9, lines 25-30). The scene layout detection is interpreted as the 
use of line direction to determine the orientation for using each sub-image. 
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c) employing an arbitration method to produce an estimate of the image 
orientation from the orientation of the detected semantic object and the detected 
orientation of the scene layout (column 9, lines 30-35). Sugiura discloses that each 
sub-image has an orientation (scene layout) calculated by using text and line direction 
information (semantic object and orientation). Then each calculated orientation has a 
reliability measure calculated and the one with the greatest reliability measure is the 
calculated orientation to be used. So the arbitration uses information from both the 
semantic object orientation and the scene layout orientation (sub-image orientation). 

With regard to claim 2, Sugiura discloses wherein the step of employing a 
semantic object detection method comprises employing as plurality of semantic object 
detectors to detect a plurality of semantic objects and their orientations (Fig. 6). 
Semantic objects are detected in each of the four sub-images in Fig. 6 in different 
detections. 

With regard to claim 3, Sugiura discloses wherein the semantic objects are 
characters in text (column 9, lines 17-23). 

With regard to claim 4, Sugiura discloses therein the scene layout detection 
method comprises the steps of: 

a) dividing the digital image into non-overlapping blocks (Fig. 6) ; 

b) computing at least one statistic for each image block (column 9, lines 26-30); 
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c) forming a feature vector by concatenating the statistics computed from the 
image blocks (column 9, lines 26-30). Sugiura discloses obtaining a reliability measure 
for each sub-image block. The reliability measure is calculated from the corresponding 
histograms. The histograms are interpreted as a feature vector. 

d) using a trained classifier to produce an estimate of the image orientation 
(column 9, lines 25-35). The trained classifier producing an estimate are the "estimates" 
or orientations calculated for each sub-image, based on these "estimates" reliability 
measures are calculated and the final orientation is decided. 

With regard to claim 5, Sugiura discloses that the invention is applied to full-color 
copiers. It is unclear what a color moment is because there is no support or definition in 
the present specification, but clearly the invention of Sugiura is in the environment of 
full-color imaging. 

With regard to claim 8, Sugiura discloses wherein the arbitration method employs 
a decision tree (Fig. 8). 

With regard to claim 9, Sugiura discloses the step of rotating the digital image to 
re-orient the digital image in an upright direction (Fig. 8, elements S1 15, S1 16, S1 17). 

With regard to claim 10, Sugiura discloses a computer program product for 
performing the method (Fig. 2). 
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With regard to claim 1 1 , Sugiura discloses a system fro performing the method 
steps as discussed in claim 1 (Figs. 2, 4 and 5). 

With regard to claims 12-15 and 18, the discussions of claims 2-5 and 8 apply 
respectively. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 6 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over 

U.S. Patent 6,798,905 to Sugiura et al. in view of IEEE document titled "Vanishing Point 

Detection by Line Clustering" by G.F. McLean and D. Kotturi hereinafter referred to as 

McLean. 

With regard to claim 6, Sugiura discloses the method of claim 1, but does not 
disclose using vanishing points to determine scene layout. However, the use of 
vanishing points to determine scene layout information is well known in the art. 

McLean discloses a vanishing pint detection method comprising the steps of: 
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a) extracting straight lines from the digital image (see abstract). 

b) computing a point of convergence (or vanishing point) from a subset of the 
extracted straight lines (see abstract), and; 

c) producing an estimate of the image orientation according to the vanishing 
point (see abstract). 

McLean "extracts straight line structure from images, develops a measure of line 
quality for each line, estimates the number of vanishing points and their approximate 
orientations, and then computes optimal vanishing point estimates through combined 
clustering and numerical optimization," taken from the abstract. McLean teaches 
determining different vanishing points using sets of line equations and using the error 
estimates of the line equations only the most likely vanishing points and their locations 
are calculated (p. 1091, left column steps 1-3). So McLean effectively teaches 
arbitration among all possible vanishing points to determine estimated vanishing points 
that are most likely definitive vanishing points. This will effectively describe the scene 
layout and will describe information about how the image is oriented. Therefore it would 
have been obvious to one of ordinary skill in the art at the time of the invention to use 
vanishing points in an image in an arbitration method of determining image orientation 
in combination with the orientation taught by Sugiura in order the arbitrate the best 
possible orientation information. 

With regard to claim 16, the discussion of claim 6 applies. 
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Claims 7 and 17 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over U.S. Patent 6,798,905 to Sugiura et al. in view of U.S. Patent 6,996,549 to Zhang 
et al. 

With regard to claim 7, Sigiura discloses the method of claim 1 but does not 
explicitly disclose the use of a Bayes Net. Bayes Nets are essentially decision trees 
with probabilities indicating the likelihood of making certain decisions. Sugiura 
discloses such a decision tree in Fig. 8 and the decision tree in Fig. 8 uses probabilities 
in the reliability degrees detected from the histograms. Sugiura does not explicitly state 
that the decisions are made with a Bayes Net. However Bayes Nets are very well 
known in the art for making decisions or indicating probabilities according to different 
likelihoods. For example Zhang teaches that image segmenting, detecting, grouping, 
and decisions can be made with any number of methods such as Bayesian networks, 
histogram thresholding, neural networks etc. Therefore it would have been obvious to 
one of ordinary skill in the art to use a Bayes Net as taught by Zhang in combination 
with the probability controlled decision tree of Sugiura shown in Fig. 8 in order to 
provide imaging detecting and decision making solutions. 

With regard to claim 17, the discussion of claim 7 applies. 
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Conclusion 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Wes Tucker whose telephone number is 571-272-7427. 
The examiner can normally be reached on 9AM-5PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on 571-272-7429. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



Wes Tucker 



3-3-06 





